Audio and video data processing device for multimedia communication via a local network set up within an asynchronous network

ABSTRACT

A device (D 1 ) is dedicated to the processing of audio and video data for multimedia communication, via an asynchronous network, between first LAN audio (TM 1 ) and video (PC 1 ) terminals and second LAN audio (TM 2 ) and video (PC 2 ) terminals. This device includes i) connection means (ML 1 ) to establish communications with the first LAN audio and video terminals and with other connection means (ML 2 ) installed in another device (D 2 ), ii) first dating means (MD 1 ) to attach a transmit time mark and an identifier to the audio and video data emitted by the first LAN audio (TM 1 ) and video (PC 1 ) terminals, before their transmission to the other connection means (ML 2 ), and to attach a receive time mark to audio and video data emitted by the other connection means (ML 2 ) and containing an identifier and a transmit time mark, and iii) processing means (MT 1 ) to determine a time difference representing the transmission time difference between the received audio and video data and presenting the same identifier, from their transmit and receive time marks, and to delay, by a value representing this time difference, the transmission of the received audio data to the first LAN audio terminal (TM 1 ) in relation to that of the received video data to the first LAN video terminal (PC 1 ).

The invention concerns the area of asynchronous networks with randomtransmission times, and more particularly multimedia communicationslinking together audio communication terminals of the LAN (Local AreaNetwork) type, and video communication terminals, also of the LAN type,within such networks.

By “audio communication terminal” is meant here any communicatingnetwork equipment capable of exchanging audio data, such as a fixed ormobile telephone for example, or a personal digital assistant (PDA).Likewise, by “video communication terminal” is meant here anycommunicating network equipment capable of exchanging video data, suchas an individual computer for example.

In asynchronous networks with random transmission times, when, forexample, two people, having activated a communication between theirfirst* audio or video LAN communication terminals via a local areanetwork (LAN) and a communication network, decide during thecommunication to communicate packets of audio or video data to eachother, via second* audio or video LAN terminals respectively,independently of the first* terminals used, offset timing usually occursbetween the packets of audio data and the packets of video data comingfrom a given user. More precisely, the audio packets generally arrivebefore the video packets, representing a nuisance for the user receivingthe data.

This offset timing results mainly from the “non-deterministic” characterof the transmission of data packets within asynchronous packet-switchednetworks, such as IP protocol networks for example. By“non-deterministic” is meant here the fact of being of a variablecharacter which is not controllable over time (i.e. of a random nature).In fact, the streams of audio data packets and the streams of video datapackets, due to their different natures, generally do not follow thesame routes within the network and generally do not have identicalpriority levels. In addition, the routes taken even by data packets ofthe same type can vary.

But the offset timing results also from differences in thecoding/decoding times of the video and audio data.

In order to attempt to improve the situation, it has been proposed thatall of the audio streams should be delayed, in an automatic manner andby a constant value, in relation to the video streams. However, thisconstant delay enables one to overcome only the non-randomtransmission-time differences, such as those introduced by thecoding/decoding operations, and these are not the most detrimental ones.

In addition, the MPEG4 standard provides a mechanism which can be usedto synchronize audio and video data, but the latter concerns onlysituations in which the audio and video data come either from a singlecommunication terminal or from two independent but synchronizedcommunication terminals.

The purpose of the invention is therefore to remedy the aforementioneddrawback. To this end, it proposes an audio and video data processingdevice for multimedia communication, via an asynchronous network withrandom transmission times, between, on the one hand, a first pairconsisting of a first audio communication terminal and a first videocommunication terminal, and on the other hand, a second pair consistingof a second audio communication terminal and a second videocommunication terminal, the said terminals being all of the LAN type;where at least the first pair consists of independent and asynchronousterminals; characterized in that it includes, in association with thisfirst pair, connection means for the setting up of:

-   -   a video link between these connection means and the video        terminal of the first pair,    -   an audio link between these connection means and the audio        terminal of the first pair,    -   a video link between these connection means and the second pair,    -   an audio link between these connection means and the second        pair.

By means of this processing device, it is possible to associate a videoterminal and an audio terminal, of the independent and asynchronous LANtype, for the setting up of a multimedia communication with another pairof terminals, thereby reducing the offset timing between the video andaudio packets received by the recipient pair of terminals. In fact, thefact that the two data streams are brought to a single point (theprocessing device) by the links ed by the processing device, bringsabout a first synchronization of these streams, before transmitting themover the links which will desynchronize them.

This first synchronization does not prevent the offset timing which thencaused by the transmission to the receiving terminals, but itfacilitates a second synchronization by making available a commonsynchronization reference for the video and the audio, while the twoterminals of the first pair are intrinsically asynchronous.

According to a preferred method of implementation, the said connectionmeans include:

-   -   First dating means arranged, on the one hand, to attach a        transmit time mark and an identifier to audio data and video        data, coming from the said first audio and video communication        terminal respectively, before their transmission to the second        pair via the said local network, and on the other hand, to        attach a receive time mark to audio and video data coming from        the is said second pair and containing an identifier and a        transmit time mark, and    -   local processing means to determine a time difference        representing the transmission time difference between the audio        data and the video data received and bearing the same        identifier, from their respective transmit and receive time        marks, and to delay, by a value representing the said time        difference, the transmission of the said audio data received at        the said first audio communication terminal in relation to the        transmission of the said video data received at the said first        video communication terminal.

By means of this double time marking, at transmission and reception, itis possible to determine the transmission time difference between theaudio and video data, for the non-deterministic part of the network thatthey follow, and then to make up for this difference by delayingtransmission of the audio data to the receiving audio communicationterminal. The second synchronization mentioned above has thus beenachieved.

The processing means are arranged preferably so as to determine a timedifference which is representative not only of the transmission timedifference but also of the coding/decoding time difference between theaudio and video data received and presenting the same identifier.

In addition, the processing means are preferably capable of determiningthe time difference not only from the transmit and receive time markingsof the audio and video data received, but also from values representingtheir respective transmission times between the second audio and videocommunication terminals which sent them and the other connection meansof the other device, in particular when the links associated with thesetimes are of the deterministic type.

As a variant or as an addition, the processing means can be arranged soas to determine the time difference not only from the transmit andreceive time markings of the audio and video data received, but alsofrom values representing their respective transmission times between theconnection means and the first* audio and video communication terminalsfor which they are intended, in particular when the links associatedwith these times are of the deterministic type.

Preferably, the processing means include a buffer memory in which theytemporarily store the audio data to be delayed before being communicatedto the connection means with a view to their transmission to the firstaudio communication terminal.

The dating means can also be arranged so as to attach data representinga priority level to the audio and video data to be transmitted to theother connection means of the other device, the priority levelassociated with the video data preferably being lower than thatassociated with the audio data.

In addition, it is advantageous that the connection means should providea function of the proxy type for both the audio and video data.

The invention also concerns an audio communication terminal, a videocommunication terminal, and a dedicated communication unit, each fittedwith a processing device of the type presented above.

The invention is particularly well suited, though not in any limitedmanner, to communications involving an asynchronous IP protocol network.

Other characteristics and advantages of the invention will appear onexamination of the detailed description below, and of the appendeddrawings, in which:

FIG. 1 schematically illustrates an example of implementation of a videocommunication terminal equipped with a device according to theinvention,

FIG. 2 schematically illustrates an example of implementation of anaudio communication terminal equipped with a device according to theinvention, and

FIG. 3 schematically illustrates an example of implementation of adedicated communication unit equipped with a device according to theinvention.

The appended drawings can serve not only to complete the invention, butalso to contribute to its description, as appropriate.

The invention concerns multimedia communications between communicationterminals via an asynchronous network with random transmission times anda local network of the LAN (Local Area Network) type, and more preciselymultimedia communications between, on the one hand, a first audiocommunication terminal and a first video communication terminal, whichare independent, non-synchronized, and of the LAN type, and on the otherhand, a second audio communication terminal and a second videocommunication terminal, which also are independent, non-synchronized andof the LAN type.

In what follows, we consider, as an example, that the local network is avirtual local network (VLAN) set up within an asynchronous IP protocolnetwork. But, the invention is neither limited to local networks of thevirtual type nor to asynchronous IP protocol networks. It concerns allasynchronous networks referred to as “packet-switched” networks, inwhich the transmission of streams of audio or video data packets is ofthe non-deterministic (or random) type.

In addition, in what follows, we consider, as an example, that eachaudio LAN communication terminal is a mobile telephone equipped so as tobe able to exchange audio data via the local (virtual) network. But theinvention is not limited to mobile telephones. It concerns allcommunicating equipment capable of exchanging audio data via the local(virtual) network, and personal digital assistants (PDA) in particular.

In addition, in what follows, we consider, as an example, that each LANvideo communication terminal is an individual computer equipped so as tobe able to exchange video data via the local (virtual) network, andpossibly coupled with a video camera. But the invention is not limitedto individual computers (whether fixed or portable). It concerns allcommunicating equipment capable of exchanging video data via the local(virtual) network.

As a consequence, in what follows, the expressions “mobile telephone”and “computer” refer to an audio communication IP terminal and a videocommunication IP terminal respectively, and the expression “localnetwork” refers to a virtual local network (VLAN) set up within the IPnetwork between the different audio and video terminals.

The invention proposes a processing device which is intended to allow toa user to receive, in a synchronized manner, for his (or her) mobiletelephone and computer (which are independent and non-synchronized),audio and video data coming from the mobile telephone and the computer(which are also independent and non-synchronized) of another user.

Reference is made to FIG. 1 in order to describe an example ofimplementation of devices according to the invention, installed in thecomputers (or PCs) of users.

In this example, a first user has a first pair consisting of a firstmobile telephone (TM1) and a first computer (PC1) containing video data(or connected to a video camera, such as a “web camera” for example,delivering video data). Likewise, a second user has a second pairconsisting of a second mobile telephone (TM2) and a second computer(PC2) containing video data (or connected to a video camera, such as a“web camera” for example, delivering video data). In addition, at leastone of these two pairs includes independent and asynchronous terminals.

Each computer (PCi, where i=1 and 2) is equipped with a processingdevice (Di) according to the invention, consisting of a link module(MLi), a dating module (MDi) and a processing module (MTi), coupled toeach other. In addition, each computer (PCi) includes a videoapplication module (MAVi) coupled to the device (Di) so as to supply itwith video data stored in a memory or coming from a video camera.

The mobile telephones (TMi) include an audio application module (notshown) which supplies audio data to an IP-LAN communication interface(ICi) allowing the exchange of audio data via the local networkestablished within the IP network (N).

The link module (MLi) of a device (Di—the first, for example) isprogrammed to activate, when its user wishes to exchange both audio andvideo data with the other user (employing methods which will be detailedlater), firstly, a link (L1) with the first mobile telephone (TM1) fortransmission of the audio data, secondly, a link (L2) with the videoapplication module (MAV1) of the computer (PC1) for transmission of thevideo data, and thirdly, two links (L3-1 and L3-2) with the link module(ML2) of the second device (D2—installed in the computer (PC2) of thesecond user) for transmission of the audio data and the video data.

Preferably, the link module (MLi) provides a function of the proxy typefor both audio and video data.

In this example, the link (L1) between the first mobile telephone (TM1)and the first computer (PC1) is set up via a switch (or “hub”) of thelocal network. This IP-LAN link (L1) is therefore of the deterministictype, which provides advance knowledge of the time (T1) necessary forthe transmission of audio data packets between the first mobiletelephone (TM1) and the first device (D1). Since the internal link (L2)is made within the first computer (PC1), it is also deterministic. Thetime (T2) necessary for the transmission of video data packets betweenthe video application module (MAV1) and the first device (D1) istherefore also known in advance.

On the other hand, the IP-LAN links (L3-1 and L3-2) between the first(D1) and second (D2) devices are set up via routes of the IP network (N)which cannot be known in advance. Since these links (L3-1 and L3-2) areof the non-deterministic (or random) type, it is therefore not possibleto have advance knowledge of the times (T3-1 and T3-2) necessary fortransmission of the audio data packets and of the video data packetsbetween the first (D1) and second (D2) devices.

When the link modules (ML1 and ML2) of the first (D1) and second (D2)devices have set up their IP-LAN links (L3-1 and L3-2), the link module(ML2), in its turn, establishes a link (L4) with the second mobiletelephone (TM2) and a link (L5) with the video application module (MAV2)of the second computer (PC2).

In this example, the link (L4) between the second mobile telephone (TM2)and the second computer (PC2) is also made via a switch (or hub) of thelocal network. This IP-LAN link (L4) is therefore also of thedeterministic type, which provides advance knowledge of the time (T4)necessary for the transmission of audio data packets between the secondmobile telephone (TM2) and the second device (D2). Since the internallink (L5) is set up within the second computer (PC2), it is alsodeterministic. The time (T5) necessary for the transmission of videodata packets between the video application module (MAV1) and the seconddevice (D2) is therefore also known in advance.

In the figures, the solid thick lines with double arrow represent thepaths taken by the streams of audio data packets, while the dotted thicklines with double arrow represent the paths taken by the streams ofvideo data packets.

The dating module (MDi) of each device (Di) is charged with two tasks. Afirst task consists of attaching a transmit time mark and a streamidentifier to each packet (or stream of packets) allowing the setting upof a correspondence between the audio and video data, each time thataudio data packets and video data packets must be transmitted from asending device or (the first (D1), for example) to a receiving device(the second (D2) for example), via the IP-LAN links (L3-1 and L3-2). Asecond task consists of attaching a receive time mark to each receivedpacket (or stream of packets), each time that audio data packets andvideo data packets have been received from a sending device (the second(D2) for example), via the IP-LAN links (L3-1 and L3-2), Time markingrefers here to an item of data representing an instant (of transmissionor reception) determined by a clock of the device (Di).

It is also preferable that the dating module (MDi) should attach datarepresenting its priority level to each audio or video data packet,taking account of its nature (audio or video). Still more preferably,the priority level associated with the video data packets is lower thanthat associated with the audio data packets. This priority level markingis preferably performed at the same time as the time marking. It is usedto provide coherence between the audio streams and the video streamsemitted from the two different sources, in order to guarantee theirrouting over the local area network (LAN) in optimal conditions and withsimilar transit times.

In addition, a VLAN identifier is attached to each packet.

The priority-level data and the VLAN identifier attached to each audioor video data packet are presented here in the form of “tags” of theIEEE 802.1 p/Q type, because of the fact that the communications are setup within a virtual local network (VLAN) specified in an IP network.

The data packets (audio or video) received by a receiving device (thefirst (D1), for example) therefore present a transmit time mark, areceive time mark, a stream identifier, a VLAN identifier, and possiblya priority level, at the output of the dating module (MD1). These markedpackets are then transmitted to the processing module (MTi) of thereceiving device (Di).

For each marked packet, the processing module (MTi) is programmed todetermine a time difference (ET) representing the transmission timedifference between audio data and video data having been transmittedvirtually simultaneously by the link module (MLj) of the sending device(Dj) and presenting the same stream identifier. This time difference(ET) is determined at least from the transmit and receive time markingsof the received audio and video packets. In addition, it is at leastrepresenting the transmission time difference (T3-2-T3-1) due to theIP-LAN links (L3-2 and L3-1) taken. However it is also preferablyrepresentative of the difference between the video coding/decoding time(TCDV) and the audio coding/decoding time (TCDA), and/or of thetransmission time difference (T2-T1) for links L2 and L1, and/or of thetransmission time difference (T5-T4) for links L5 and L4.

If possible, the time difference determined is representative of all theaforementioned parameters. In this case, one gets the followingrelation:ET=T 3-2−T 3-1+TCDV−TCDA+T 2−T 1+T 5−T 4

This relation can be simplified when one considers that the transmissiontimes (T2 and T5) are negligible due to the fact that here the device(Di) and video application module (MAVi) are installed in a singlecomputer (PCi). In this case, the relation can be rewritten:ET=ΔT 3+TCDV−TCDA+T 1+T 4

-   -   where, ΔT3=T3-2−T3-1.

Since the audio coding/decoding time (TCDA) and video coding/decodingtime (TCDV) are constant and known, just like T1 and T4, the processingmodule (MTi) therefore only has to determine ΔT3 in order to ascertainthe time difference (ET). To this end, it only has to determine, foreach received audio packet, a received video packet presenting atransmit time marker which is virtually identical (or in other words avirtually identical instant of transmission) and the same streamidentifier. It then determines the time (T3-1) equal to the timedifference between the transmit and receive time markings of the audiopacket, and the time (T3-2) equal to the time difference between thetransmit and receive time markings of the video packet. The processingmodule (MTi) then only has to perform the subtraction T3-2−T3-1 in orderto obtain ΔT3.

Since the transmission of an audio packet is usually faster than that ofa video packet, it is not necessary to determine the time difference(ET) for the video packet. In addition it is equal, all but in sign, tothat of the associated audio packet.

Once in possession of the time difference (ET) associated with an audiopacket, the processing module (MTi) determines a delay, representingthis time difference, to be applied to it before transmitting it to themobile telephone (TMi) for which it is intended. The video packets aretransmitted immediately, that is without delay to the link module (MLi)in order to communicate it via the link (L2 or L5) to the recipientvideo application module (MAVi) so that they can be displayed.

The delayed audio packets are preferably stored in a buffer memory (Mi)for a time equal to the calculated delay, and then they are transmittedto the link module (MLi) in order that it can communicate them via thelink (L1 or L4) to the mobile telephone (TMi) in order to be reproducedby its receiver.

The use of a device according to the invention can occur in at least twoways.

A first way concerns the situation in which the two users have set up anaudio IP-LAN link (L6) between their mobile telephones (TM1 and TM2),via their IP-LAN communication interfaces (IC1 and IC2) and the localnetwork instituted in the IP network (N). Each mobile telephone (TMi)then knows the IP address of the other mobile telephone (TMj).

At a given instant, the users decide to communicate video data to eachother via their computers (PC1 and PC2), at the same time as audio data.

The mobile telephones (TM1 and TM2) then exchange the IP addresses ofthe computers (PC1 and PC2) with which they are going to be associatedrespectively. These IP addresses have been supplied to them by theirrespective users or by a call server, or have been extracted from amemory of each mobile telephone (TMi). Each mobile telephone (TMi) thenestablishes an IP-LAN link with the associated computer (PCi), in orderto indicate to it that it wishes to establish a multimedia (audio/video)communication with another mobile telephone (TMj) and another computer(PCJ), for which it is supplying the received IP addresses. Eachcomputer (PCi) then activates the resources necessary for thiscommunication. Since the device (Di) is alerted by the computer (PCi),in which it is installed here, its link module (MLi) then establishesthe IP-LAN links (L1 or L4) and internal links (L2 or L5) with themobile telephone (TMi) and the video application module (MAVi), as wellas the IP-LAN audio links (L3-1) and video links (L3-2) with the linkmodule (MLj) of the other device (Dj).

Once these links have been set up, the mobile telephone (TMi-TM1 forexample) and the video application module (MAVi-MAV1 for example) canthen simultaneously communicate their audio and video data packets tothe link module (MLi-ML1 for example), which transmits them to thedating module (MDi-MD1 for example) to which it is coupled. As theyarrive, the audio and video packets receive their transmit time mark,their stream identifier, and their VLAN identifier (and possibly thepriority-level data) from the dating module (MD1). Then the markedpackets are again transmitted to the link module (ML1) so that it cantransmit them to the link module (ML2) of the receiving device (D2) overthe IP-LAN audio (L3-1) and video (L3-2) links, according to theirnature (audio or video).

On receipt of these packets, the link module (ML2) transmits them to thedating module (MD2) to which it is coupled. As they arrive, the audioand video packets obtain their receive time mark from the dating module(MD2). Then the marked packets are transmitted to the processing moduleMT2 which determines the associated time difference (ET) andcorresponding delay. The audio packets are then stored in the buffermemory (M2) of the processing module (MT2), while the video packets areimmediately transmitted to the link module (ML2) so that it can transmitthem to the video application module (MAV2) of the second computer(PC2), via the internal link (L5). Once the delay associated with anaudio packet has expired, the processing module (MT2) extracts it fromthe buffer memory (M2) and then sends it to the link module (ML2) sothat it can transmit it to the second mobile telephone (TM2) via theIP-LAN link (L4).

An identical processing mechanism is implemented in respect of the firstdevice (D1) for the audio and video packets emitted from the seconddevice (D2) and intended for the first mobile telephone (TM1) and thevideo application module (MAV1) of the first computer (PC1).

A second way concerns the situation in which the two users have set upan IP-LAN video link (L7) between their computers (PC1 and PC2), viatheir IP-LAN communication interfaces (not shown) and the local networkinstituted in the IP network (N). Each computer (PCi) then knows the IPaddress of the other computer (PCj).

At a given instant, the users decide to communicate audio data to eachother via their mobile telephones (TM1 and TM2), at the same time asvideo data.

The computers (PC1 and PC2) then exchange the IP addresses of the mobiletelephones (TM1 and TM2) with which they are going to be associatedrespectively. These IP addresses have been supplied to them by theirrespective users or by a call server, or have been extracted from amemory of each computer (PCi). Each computer (PCi) then establishes anIP-LAN link with the associated mobile telephone (TMi) in order toindicate to it that it wishes to establish a multimedia (audio/video)communication with another mobile telephone (TMj) and another computer(PCj), for which it is supplying the received IP addresses. What followsis then identical to that which has been described above for the firstway.

The above is a description, with reference to FIG. 1, of an example ofimplementation in which the link between the mobile telephone (TMi) andthe computer (PCi) is of the IP-LAN type. However it is possible toenvisage a variant in which this link is of the Ethernet type. In thiscase, the mobile telephone (TMi) and the computer (PCi) includecommunication interfaces of the IP-LAN type and of the Ethernet type.

Reference is now made to FIG. 2 in order to describe an example ofimplementation of devices according to the invention, installed in themobile telephones (TMi) of users.

Each mobile telephone (TMi) is equipped with a processing device (Di)according to the invention, which is virtually identical to thatdescribed previously with reference to FIG. 1, and coupled to an audioapplication module (MMi) and to an interface of the Ethernet type (notshown).

In addition, each computer (PCi) includes a video application module(MAVi) coupled to an Ethernet interface (IEi) allowing it to exchangevideo data directly with the mobile telephone (TMi) without traversingthe IP network (N). arrangement (Di-the first, for example) is herecharged, when its user wishes to exchange both audio data and video datawith the other user (employing methods which will be detailed later) toestablish, firstly, an internal link (L′1 or L′4) with the audioapplication module (MAAi), secondly, an Ethernet link (L′2 or L′5) withthe video application module (MAVi) of the computer (PCi) fortransmission of the video data via the Ethernet interfaces, and thirdly,two IP-LAN links (L′3-1 and L′3-2) with the link module (MLj) of theother device (Dj-installed in computer PCJ) for transmission of theaudio data and the video data via IP-LAN interfaces (not shown).

The Ethernet link (L′2 or L′5) runs directly from the computer (PC1 orPC2) to the mobile telephone (TM1 or TM2) via their Ethernet interfaces,so that it is of the deterministic type. Since the internal link (L′1 orL′4) is set up within the mobile telephone (TM1 or TM2), it is also ofthe deterministic type. On the other hand, since the IP-LAN links (L′3-1and L′3-2) between the first (D1) and second (D2) devices are set up viaroutes over the IP network (N) which cannot be known in advance, theyare of the non-deterministic (or random) type.

Calculation of the time differences (ET) by a processing module (MTi) isidentical to that presented above with reference to FIG. 1. Only thesimplified relation of this time difference (ET) is different, and sincehere it is the transmission times (T′1 and T′4) which are negligible dueto the fact that the device (Di) and the audio application module (MAAi)are installed in a single mobile telephone (TMi). In this case, it canbe re-written as:ET=ΔT′3+TCDV−TCDA+T′2+T′5

-   -   where, ΔT′3=T′3-2−T′3-1.

In this example, use of the devices (Di) can again be effected in atleast two ways.

A first way concerns the situation in which the two users have set up anaudio IP-LAN link (L′6) between their mobile telephones (TM1 and TM2),via their IP-LAN communication interfaces and the local networkinstituted in the IP network (N). Each mobile telephone (TMi) then knowsthe IP address of the other mobile telephone (TMj).

At a given instant, the users decide to communicate video data to eachother via their computers (PC1 and PC2), at the same time as audio data.

The mobile telephones (TM1 and TM2) then exchange the IP addresses ofthe computers (PC1 and PC2) with which they are going to be associatedrespectively, via the IP-LAN links (L′6). These IP addresses have beensupplied to them by their respective users or by a call server, or havebeen extracted from a memory of each mobile telephone (TMi). Each mobiletelephone (TMi) then establishes an Ethernet link with the associatedcomputer (PCi), in order to indicate to it that it wishes to establish amultimedia (audio/video) communication with another computer (PCj), forwhich it is supplying the received IP address, via the device (Di) thatit accommodates.

Each computer (PCi) then activates the resources necessary for thiscommunication. Since the device (Di) is alerted by the mobile telephone(TMi), in which it is installed here, its link module (MLi) thenestablishes the internal link (L′1 or L′4) with the audio applicationmodule (MAAi), the Ethernet link (L′2 or L′5) with the computer (PCi),as well as the IP-LAN audio and video links (L′3-1 and L′3-2) with thelink module (MLj) of the other device (Dj).

Once these links have been set up, the audio application module (MMi) ofthe mobile telephone (TMi-TM1 for example) and the video applicationmodule (MAVi) of the computer (PCi-PC1 for example) can thensimultaneously communicate their audio and video data packets, via theinternal links (L′1) and Ethernet links (L′2), to the link module(MLi-ML1 for example), which transmits them to the dating module(MDi-MD1 for example) to which it is coupled. As they arrive, the audioand video packets receive their transmit time mark, their streamidentifier and their VLAN identifier (and possibly the priority-leveldata) from the dating module (MD1). Then the marked packets are againtransmitted to the link module (ML1) so that it can transmit them to thelink module (ML2) of the receiving device (D2) over the IP-LAN audio andvideo links (L′3-1 and L′3-2) according to their nature (audio orvideo).

On receipt of these packets, the link module (ML2) transmits them to thedating module (MD2) to which it is coupled. As they arrive, the audioand video packets obtain their receive time mark from the dating module(MD2). Then the marked packets are transmitted to the processing module(MT2) which determines the associated time difference (ET) and thecorresponding delay. The audio packets are then stored in the buffermemory (M2) of the processing module (MT2), while the video packets areimmediately transmitted to the link module (ML2) so that it can transmitthem, via the Ethernet link (L′5), to the video application module(MAV2) of the second computer (PC2). Once the delay associated with anaudio packet has expired, the processing module (MT2) extracts it fromthe buffer memory (M2) and then transmits it to the link module (ML2) sothat it, in turn, can transmit it to the audio application module (MAA2)of the second mobile telephone (TM2), via the internal link (L′4).

An identical processing mechanism is implemented in respect of the firstdevice (D1) for the audio and video packets emitted from the seconddevice (D2) and intended for the first mobile telephone (TM1) and thevideo application module (MAV1) of the first computer (PC1).

A second way concerns the situation in which the two users have set upan IP-LAN video link (L′7) between their computers (PC1 and PC2), viatheir IP-LAN communication interfaces (IC1 and IC2) and the localnetwork instituted in the IP network (N). Each computer (PCi) then knowsthe IP address of the other computer (PCj).

At a given instant, the users decide to communicate audio data to eachother via their mobile telephones (TM1 and TM2), at the same time asvideo data.

The computers (PC1 and PC2) then exchange the IP addresses of the mobiletelephones (TM1 and TM2) with which they are going to be associatedrespectively. These IP addresses have been supplied to them by theirrespective users or by a call server, or have been extracted from amemory of each computer (PCi). Each computer (PCi) then establishes anIP-LAN link with the associated mobile telephone (TMi), in order toindicate to its device (Di) that it wishes to establish a multimedia(audio/video) communication with another mobile telephone (TMj) andanother computer (PCj), for which it is supplying the received IPaddresses. What follows is then identical to that which has beendescribed above for the first way.

The above is a description, with reference to FIG. 2, of an example ofimplementation in which the link between the mobile telephone (TMi) andthe computer (PCi) is of the Ethernet type. However it is possible toenvisage a variant in which this link is of the IP-LAN type. In thiscase, the mobile telephone (TMi) and the computer has only IP-LANcommunication interfaces.

Reference is now made to FIG. 3, in order to describe an example ofimplementation of devices according to the invention, installed indedicated communication units (Bi).

Each mobile telephone (TMi) is here equipped with an audio applicationmodule (MAAi) coupled to an IP-LAN interface (not shown) and to anEthernet interface (IETi). In addition, each computer (PCi) includes avideo application module (MAVi) coupled to an IP-LAN communicationinterface (not shown) and to an Ethernet interface (IEPi).

Each communication unit (Bi) includes a processing device (Di) accordingto the invention, which is virtually identical to that describedpreviously with reference to FIGS. 1 and 2, and coupled to communicationinterfaces. In the illustrated example, the unit (Bi) is more preciselyequipped with Ethernet interfaces (not shown), connected to the Ethernetinterfaces (IETi and IEPi) of the mobile telephone (TMi) and of thecomputer (PCi), and IP-LAN interfaces (not shown) coupled to the IPnetwork (N).

However the dedicated unit (Bi) can also take the form of acommunication card installed in a server of the IP network (N) andconsisting only of a device (Di) coupled to an IP-LAN interface.

The link module (MLi) of a device (Di—the first, for example) is hereprogrammed, when its user wishes to exchange both audio data and videodata with the other user (employing methods which will be detailedlater) to establish, firstly, an Ethernet link (L′1 or L′4) with theaudio application module (MAAi) of the mobile telephone (TMi) fortransmission of the audio data via the Ethernet interfaces, secondly, anEthernet link (L′2 or L′5) with the video application module (MAVi) ofthe computer (PCi) for transmission of the video data via the Ethernetinterfaces, and thirdly, two IP-LAN links (L′3-1 and L′3-2) with thelink module (MLj) of the other device (Dj-installed in computer PCj) fortransmission of the audio and video data via the IP-LAN interface (notshown).

The Ethernet link (L′1) runs directly from the mobile telephone (TMi) tothe unit (Bi) via their Ethernet interfaces, so that it is of thedeterministic type. Likewise, the Ethernet link (L′2) runs directly fromthe computer (PCi) to the unit (Bi) via their Ethernet interfaces, sothat it is of the deterministic type. On the other hand, the IP-LANlinks (L′3-1 and L′3-2) between the first (Dl) and second (D2) devicesset up over routes of the IP network (N) which cannot be known inadvance, they are of the non-deterministic (or random) type.

Calculation of the time differences (ET) by a processing module (MTi) isidentical to that presented above with reference to FIG. 1. When thetransmission times (T″1 and T″2 or T″4 and T″5) for the Ethernet links(L′1) and L′2 or L′4 and L′5) are virtually identical, the simplifiedrelation of the time difference (ET) can be re-written as follows:ET=ΔT″ 3+TCDV−TCDA

-   -   where, ΔT″3=T″3-2)−T″3-1.

In this example, use of the devices (Di) can again be effected in atleast two ways.

A first way concerns the situation in which the two users have set up anaudio IP-LAN link (L′6) between their mobile telephones (TM1 and TM2),via their IP-LAN communication interfaces and the local networkinstituted in the IP network (N). Each mobile telephone (TMi) then knowsthe IP address of the other mobile telephone (TMj).

At a given instant, the users decide to communicate video data to eachother via their computers (PC1 and PC2), at the same time as audio data.

The mobile telephones (TM1 and TM2) then exchange the IP addresses ofthe computers (PC1 and PC2) with which they are going to be associatedrespectively, via the IP-LAN links (L′6). These IP addresses have beensupplied to them by their respective users or by a call server, or havebeen extracted from a memory of each mobile telephone (TMi). Each mobiletelephone (TMi) then establishes an Ethernet link with the unit (Bi) inorder to indicate to it that it wishes to establish a multimedia(audio/video) communication with another computer (PCj) and the mobiletelephone (TMj), for which it is supplying the IP addresses. The unit(Bi) then establishes an Ethernet link with the computer (PCi) in orderto indicate to it that the mobile telephone (TMi) wishes to establish amultimedia (audio/video) communication with the computer (PCJ) and themobile telephone (TMj). The computer (PCi) then activates the resourcesnecessary for this communication. The device (Di) of the unit (Bi) thenestablishes, by means of its link module (MLi), the Ethernet link (L′1)(or L′4) with the audio application module (MAAi) of the mobiletelephone (TMi), the Ethernet link (L′2) (or L′5) with the videoapplication module (MAVi) of the computer (PCi), and the IP-LAN audioand video links (L′3-1 and L′3-2) with the link module (MLj) of theother device (Dj).

Once these links have been set up, the audio application module (MAAi)of the mobile telephone (TMi-TM1 for example) and the video applicationmodule (MAVi) of the computer (PCi-PC1 for example) can thensimultaneously communicate their audio and video data packets, via theEthernet links (L′1 and L′2), to the link module (MLi-ML1 for example),which transmits them to the dating module (MDi-MD1 for example) to whichit is coupled. As they arrive, the audio and video packets receive theirtransmit time mark, their stream identifier and their VLAN identifier(and possibly the priority-level data from the dating module (MD1). Thenthe marked packets are again transmitted to the link module (ML1) sothat it can transmit them to the link module (ML2) of the receivingdevice (D2) over the IP-LAN audio and video links (L′3-1 and L′3-2)according to their nature (audio or video).

On receipt of these packets, the link module (ML2) transmits them to thedating module (MD2) to which it is coupled. As they arrive, the audioand video packets obtain their receive time mark from the dating module(MD2). Then the marked packets are transmitted to the processing module(MT2) which determines the associated time difference (ET) andcorresponding delay. The audio packets are then stored in the buffermemory (M2) of the processing module (MT2), while the video packets areimmediately transmitted to the link module (ML2) so that it can transmitthem, via the Ethernet link (L′5), to the video application module(MAV2) of the second computer (PC2). Once the delay associated with anaudio packet has expired, the processing module (MT2) extracts it fromthe buffer memory (M2) and then transmits it to the link module (ML2) sothat it, in turn, can transmit it to the audio application module (MAA2)of the second mobile telephone (TM2), via the Ethernet link (L′4).

An identical processing mechanism is implemented in respect of the firstdevice (D1) for the audio and video packets emitted from the seconddevice (D2) and intended for the first mobile telephone (TM1) and forthe video application module (MAV1) of the first computer (PC1).

A second way concerns the situation in which the two users have set upan IP-LAN video link (L′7) between their computers (PC1 and PC2), viatheir IP-LAN communication interfaces (IC1 and IC2) and the localnetwork instituted in the IP network (N). Each computer (PCi) then knowsthe IP address of the other computer (PCj).

At a given instant, the users decide to communicate audio data to eachother via their mobile telephones (TM1 and TM2), at the same time asvideo data.

The computers (PC1 and PC2) then exchange the IP addresses of the mobiletelephones (TM1 and TM2) with which they are going to be associatedrespectively. These IP addresses have been supplied to them by theirrespective users or by a call server, or have been extracted from amemory of each computer (PCi). Each computer (PCi) then establishes anIP-LAN link with the associated mobile telephone (TMi) in order toindicate to its device (Di) that it wishes to establish a multimedia(audio/video) communication with another mobile telephone (TMj) andanother computer (PCj), for which it is supplying the received IPaddresses. What follows is then identical to that which has beendescribed above for the first way.

The above is a description, with reference to FIG. 3, of an example ofimplementation in which the links between the unit (Bi) and the mobiletelephone (TMi), on the one hand, and between the unit (Bi) and thecomputer (PCi), on the other hand, are of the Ethernet type. However itis possible to envisage a first variant in which these two links are ofthe IP-LAN type. In this case, the unit (Bi), the mobile telephone(TMi), and the computer (PCi) have only IP-LAN communication interfaces.On can also envisage a second variant in which one of the two links isof the Ethernet type while the other link is of the IP-LAN type.

The processing device (Di) according to the invention can be implementedin the form of electronic circuits, software (computer) modules, or acombination of circuits and software. It is preferable to implement itby means of software modules due to the fact that it is intended toexecute processes for application protocols whose level is above levelfour of the OSI layer model.

The invention is not limited to the methods of implementation of theprocessing device, of the audio communication terminal, of the videocommunication terminal and of the dedicated communication unit describedabove as an example only, but covers all variants which can be envisagedby the professional engineer in the context of the following claims

Thus examples of implementation suitable for communication via anasynchronous IP protocol network have been described. However theinvention is not limited to these asynchronous networks only. Itconcerns, in a general manner, all of the networks referred to as“packet-switched” networks, in which the transmission of streams ofaudio or video data packets is of the non-deterministic type.

1. Audio and video data processing device (D1) for multimediacommunication (L3-1, L3-2), via an asynchronous network (N) with randomtransmission times, between, on the one hand, a first pair consisting ofa first audio communication terminal (TM1) and a first videocommunication terminal (PC1), and on the other hand, a second pairconsisting of a second audio communication terminal (TM2) and a secondvideo communication terminal (PC2), the said terminals (TMi, PCi) beingall of the LAN type, where at least the first pair (TM1, PC1) consistsof independent and asynchronous terminals, characterized in that itincludes, in association with this first pair, connection means (ML1)for the setting up of: a video link (L2) between these connection means(ML1) and the video terminal (PC1) of the first pair, an audio link (L1)between these connection means (ML1) and the audio terminal (TM1) of thefirst pair, a video link (L3-2) between these connection means (ML1) andthe second pair (TM2, PC2), an audio link (L3-1) between theseconnection means (ML1) and the second pair (TM2, PC2).
 2. Deviceaccording to claim 1, characterized in that the said connection means(ML1) include: first* dating means (MD1) arranged, on the one hand, toattach a transmit time mark and an identifier to audio and video data,coming from the first audio (TM1) and video (PC1) communication terminalrespectively, before their transmission to the second pair via the saidlocal network, and on the other hand, to attach a receive time mark tothe audio and video data coming from the said second pair and containingan identifier and a transmit time mark, and their own processing means(MT1) to determine a time difference (ET) representing the transmissiontime difference between the received audio and video data and presentingthe same identifier, from their respective transmit and receive timemarks, and to delay, by a value representing the said time difference(ET), the transmission of the said received audio data at the said firstaudio communication terminal (TM1) in relation to the transmission ofthe said received video data at the said first video communicationterminal (PC1).
 3. Device according to claim 2, characterized in thatthe said processing means (MT1) are arranged so as to determine a timedifference (ET) representing the said transmission time difference and acoding and decoding time difference between the received audio and videodata and presenting the same identifier.
 4. Device according to claim 2,characterized in that the said processing means (MT1) are arranged so asto determine the said time difference (ET) from the transmit and receivetime markings of the said received audio and video data, and from valuesrepresenting their respective transmission times between the secondaudio (TM2) and video (PC2) communication terminals of the second pairwhich transmitted them, and other connection means (ML2) associated withthe second pair.
 5. Device according to claim 4, characterized in thatthe said links between the second audio (TM2) and video (PC2)communication terminals and the other connection means (ML2) associatedwith the second pair are of the “deterministic” type.
 6. Deviceaccording to claim 2, characterized in that the said processing means(MT1) are arranged so as to determine the said time difference (ET) fromthe transmit and receive time markings of the said received audio andvideo data, and from values representing their respective transmissiontimes between the said connection means (ML1) and the first audio (TM1)and video (PC1) communication terminals for which they are intended. 7.Device according to claim 6, characterized in that the said linksbetween the said connection means (ML1) and the first audio (TM1) andvideo (PC1) communication terminals are of the “deterministic” type. 8.Device according to claim 2, characterized in that the said dating means(MD1) are also arranged so as to attach data, representing a prioritylevel, to the said audio data and video data to be transmitted to theother connection means (ML2).
 9. Device according to claim 9,characterized in that the said priority level associated with the saidvideo data is lower than the said priority level associated with thesaid audio data.
 10. Device according to claim 1, characterized in thatthe said connection means (ML1) provide a function of the proxy type forthe said audio data and video data.
 11. Audio communication terminal ofthe LAN type (TMi), characterized in that it includes a processingdevice (Di) according to claim
 1. 12. Video communication terminal ofthe LAN type (PCi), characterized in that it includes a processingdevice (Di) according to claim
 1. 13. Communication unit (Bi),characterized in that it includes a processing device (Di) according toclaim 1.